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I CLAIM: 

1. A method of encoding a video stream, the method 
comprising the steps of: 

receiving the video stream; 
5 generating a base bitstream comprising one or 

more base video object planes (VOPs) using the video 
stream, each base VOP being associated with a base 
presentation time stamp (PTS) and a base decoding time 
stamp (DTS) ; and 

10 generating a first enhancement bitstream 

comprising one or more first enhancement VOPs using the 
§3 video stream, each first enhancement VOP being associated 

•~ with a corresponding base VOP, a first DTS and a first PTS, 

IS 

kU wherein the first DTS and the first PTS 

m 

15 associated with each first enhancement VOP are selected to 
be equal to one another, the first PTS associated with each 
.~ first enhancement VOP is selected to be equal to the base 

~ PTS associated with its corresponding base VOP, and the 

first DTS associated with each first enhancement VOP is 
20 selected to be equal to the base DTS associated with one of 
the base VOPs. 



m 



2. The method of encoding the video stream according 
to claim 1, the method further comprising the step of 

25 generating a second enhancement bitstream comprising one or 
more second enhancement VOPs using the video stream, 
wherein each second enhancement VOP is associated with two 
corresponding base VOPs, a second DTS and a second PTS, and 
wherein the second DTS and the second PTS associated with 

30 each second enhancement VOP are selected to be equal to one 
another . 
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3 . The method of encoding the video stream according 
to claim 2, wherein the first DTS is selected to be 
different from any of the second DTSs . 



5 4 . The method of encoding the video stream according 

to claim 3, wherein the second DTS associated with each 
second enhancement VOP represents an interval that is right 
after the later of the two intervals represented by the two 
base DTSs associated with its two corresponding base VOPs . 

10 

5. The method of encoding the video stream according 
D to claim 1, wherein the encoding comprises MPEG-4 encoding. 

m 

)f m 6. The method of encoding the video stream according 

all 

O 15 to claim 2, wherein the encoding comprises fine granularity 
;^ scalability (FGS) encoding. 



in 7 . The method of encoding the video stream according 

*z to claim 6, wherein the first enhancement bitstream 

in 

p 20 comprises FGS bitstream and the first enhancement VOPs 

f ~ comprise FGS VOPs. 

8 . The method of encoding the video stream according 
to claim 7, wherein the second enhancement bitstream 

25 comprises FGS temporal scalability (FGST) bitstream and the 
second enhancement VOPs comprise FGST VOPs. 

9. The method of encoding the video stream according 
to claim 2, the method further comprising the step of 

30 combining the first and second enhancement bitstreams to 
generate a single enhancement bitstream. 
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10. The method of encoding the video stream according 
to claim 1, the method further comprising the steps of 
packetizing the base bitstream and the first enhancement 
bitstream, and multiplexing the packetized bitstreams to 
generate a transport stream. 

11. A method of decoding a multiplexed bitstream to 
generate a video stream, the method comprising the steps 
of: 

receiving the multiplexed bitstream; 

demultiplexing and depacketizing the multiplexed 
bitstream to generate a base bitstream and a first 
enhancement bitstream; 

decoding the base bitstream to generate one or 
more base video object planes (VOPs), each base VOP being 
associated with a base presentation time stamp (PTS) and a 
base decoding time stamp (DTS) ; 

decoding the first enhancement bitstream to 
generate one or more first enhancement VOPs, each first 
enhancement VOP being associated with a corresponding base 
VOP, a first DTS and a first PTS; and 

presenting the first enhancement VOPs and the 
base VOPs to be displayed, 

wherein each first enhancement VOP is decoded and 
presented at the same time unit, and wherein each first 
enhancement VOP and its corresponding base VOP are 
presented at the same time unit. 

12. The method of decoding the multiplexed bitstream 
according to claim 11, wherein the demultiplexing and 
depacketizing step further generates a second enhancement 
bitstream, method further comprising the step of decoding 
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the second enhancement bitstream to generate one or more 
second enhancement VOPs, wherein each second enhancement 
VOP is associated with two corresponding base VOPs, a 
second DTS and a second PTS, and wherein each second 
5 enhancement VOP is decoded and presented at the same time 
unit . 

13. The method of decoding the multiplexed bitstream 
according to claim 12, wherein each second enhancement VOP 

10 is decoded right after the second of the two corresponding 
base VOPs has been decoded, unless this would cause the 
^ second enhancement VOPs to be decoded out of display order, 

^0 in which case, the second enhancement VOPs are decoded in 

IS 

ifj the display order. 

55 I 

v i is 

y 

14. The method of decoding the multiplexed bitstream 
'** according to claim 11 wherein the decoding comprises MPEG-4 
Q decoding. 

m 
iu 

*•*} 20 15. The method of decoding the multiplexed bitstream 

U 

}=& according to claim 12, wherein the decoding comprises fine 

granularity scalability (FGS) decoding. 

16. The method of decoding the multiplexed bitstream 
25 according to claim 15, wherein the first enhancement 

bitstream comprises FGS bitstream and the first enhancement 
VOPs comprise FGS VOPs. 

17. The method of decoding the multiplexed bitstream 
30 according to claim 16, wherein the second enhancement 

bitstream comprises FGS temporal scalability (FGST) 
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bitstream and the second enhancement VOPs comprise FGST 
VOPs. 

18. The method of decoding the multiplexed bitstream 
5 according to claim 11, wherein the multiplexed bitstream is 

an MPEG-4 Transport stream. 

19. A video encoding system for generating a base 
bitstream and one or more enhancement bitstreams using a 

10 video stream, the video encoding system comprising: 

a base encoder for receiving the video stream and 
for generating the base bitstream using the video stream, 

O 

~p the base bitstream comprising one or more base video object 

planes (VOPs); 

l U 15 an enhancement encoder for receiving processed 

video data from the base encoder and for generating a first 
enhancement bitstream using the processed video data, the 



s— 

o 

s 



Q first enhancement bitstream comprising one or more first 

i'fi 

enhancement VOPs, each first enhancement VOP being 

§ y 

111 20 associated with a corresponding base VOP; and 
i*^ a multiplexer for time stamping each base VOP 

with a base decoding time stamp (DTS) and a base 
presentation time stamp (PTS) , for time stamping each first 
enhancement VOP with a first DTS and a first PTS, for 
25 packetizing the base bitstream and the first enhancement 
bitstream into packets, and for multiplexing the packets to 
generate a multiplexed bitstream, 

wherein the first DTS and the first PTS 
associated with each first enhancement VOP are selected to 
30 be equal to one another, the first PTS associated with each 
first enhancement VOP is selected to be equal to the base 
PTS associated with its corresponding base VOP, and the 
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first DTS associated with each first enhancement VOP is 
selected to be equal to the base DTS associated with one of 
the base VOPs . 

5 20. The video encoding system according to claim 19, 

wherein the enhancement encoder generates a second 
enhancement bitstream using the processed video data, . the 
second enhancement bitstream comprises one or more second 
enhancement VOPs, and each second enhancement VOP is 
10 associated with two corresponding base VOPs, 

wherein the multiplexer time stamps each second 
enhancement VOP with a second DTS and a second PTS, 

U 

%(% packetizes the second enhancement bitstream into second 

^ packets, and multiplexes the second packets with the 

ft! 15 packets for the base bitstream and the first enhancement 

bitstream to generate the multiplexed bitstream, and 
M wherein the second DTS and the second PTS 

Q associated with each second enhancement VOP are selected to 

III be equal to one another. 

(II 20 

o 

ij, 21. The video encoding system according to claim 20, 

wherein the first DTS is selected to be different from any 
of the seconjd DTSs. 

25 22. The video encoding system according to claim 21, 

wherein the second DTS associated with each second 
enhancement VOP represents an interval that is right after 
the later of the two intervals represented by the two base 
DTSs associated with its two corresponding base VOPs. 

30 
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23. The video encoding system according to claim 19, 
wherein the encoding comprises MPEG-4 encoding, and the 
multiplexed bitstream is an MPEG-4 Transport stream. 



• 



5 24. The video encoding system according to claim 20, 

wherein the encoding comprises fine granularity scalability 
(FGS) encoding. 

25. The video encoding system according to claim 24, 
10 wherein the first enhancement bitstream comprises FGS 
bitstream and the first enhancement VOPs comprise FGS VOPs . 

O 

bQ 26. The video encoding system according to claim 25, 

wherein the second enhancement bitstream comprises FGS 
15 temporal scalability (FGST) bitstream and the second 
fl enhancement VOPs comprise FGST VOPs. 

13 

0 27. The video encoding system according to claim 20, 

if! 

i*« wherein the first and second enhancement bitstreams are 

1^ 20 combined to generate a single enhancement bitstream. 

O 

28. The video encoding system according to claim 19, 
wherein each VOP comprise a plurality of bit planes. 

25 29. The video encoding system according to claim 19, 

wherein the base encoder performs discrete cosine transform 
(DCT) on the video stream to generate DCT coefficients, and 
wherein the DCT coefficients are provided as the processed 
video data to the enhancement encoder. 

30 
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30. A video decoding system for generating a base 
layer video and an enhancement video using a multiplexed 
bitstream, the video decoding system comprising: 

a demultiplexer for demultiplexing and 
5 depacketizing the multiplexed bitstream to generate a base 
bitstream and a first enhancement bitstream; 

a base decoder for decoding the base bitstream to 
generate one or more base video object planes (VOPs) , each 
base VOP being associated with a base presentation time 
10 stamp (PTS) and a base decoding time stamp (DTS) ; and 

an enhancement decoder for decoding the first 
enhancement bitstream to generate one or more first 

^0 enhancement VOPs, each first enhancement VOP being 

138 

.g associated with a corresponding base VOP, a first DTS and a 

jj~ 15 first PTS, 

2 j 

m wherein each first enhancement VOP is decoded and 

]~ presented at the same time unit, and wherein each first 

O enhancement VOP and its corresponding base VOP are 

IIP 

jy presented at the same time unit. 

!!J 20 

31. The video decoding system according to claim 30, 
wherein the demultiplexer generates a second enhancement 
bitstream, and the enhancement decoder decodes the second 
enhancement bitstream to generate one or more second 

25 enhancement VOPs, and wherein each second enhancement VOP 
is associated with two corresponding base VOPs, a second 
DTS and a second PTS, and wherein each second enhancement 
VOP is decoded and presented at the same time unit. 

30 32. The video decoding system according to claim 31, 

wherein each second enhancement VOP is decoded right after 
the second of the two corresponding base VOPs has been 
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decoded, unless this would cause the second enhancement 
VOPs to be decoded out of display order, in which case, the 
second enhancement VOPs are decoded in the display order. 

5 33. The video decoding system according to claim 31 

wherein the base decoder comprises one or more frame 
buffers for storing partially decoded base bitstream and 
the enhancement decoder comprises one or more frame buffers 
for storing partially decoded second enhancement bitstream, 
10 wherein not more than a total of three frame buffers are 
used at a time for decoding and presenting all of the base 
bitstream and the first and second enhancement bitstreams. 

34. The video decoding system according to claim 30 
15 wherein the decoding comprises MPEG-4 decoding. 

35. The video decoding system according to claim 31, 
wherein the decoding comprises fine granularity scalability 
(FGS) decoding. 



36. The video decoding system according to claim 35, 
wherein the first enhancement bitstream comprises FGS 
bitstream and the first enhancement VOPs comprise FGS VOPs. 

25 37. The video decoding system according to claim 36, 

wherein the second enhancement bitstream comprises FGS 
temporal scalability (FGST) bitstream and the second 
enhancement VOPs comprise FGST VOPs. 

30 38. The video decoding system according to claim 30, 

wherein the multiplexed bitstream is an MPEG-4 Transport 
stream. 
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